<!DOCTYPE html>
<html>

<head>
  <title>Quarkus - Measuring Performance</title>
  <script id="adobe_dtm" src="https://www.redhat.com/dtm.js" type="text/javascript"></script>
  <script src="/assets/javascript/highlight.pack.js" type="text/javascript"></script>
  <META HTTP-EQUIV='Content-Security-Policy' CONTENT="default-src 'none'; script-src 'self' 'unsafe-eval' 'sha256-ANpuoVzuSex6VhqpYgsG25OHWVA1I+F6aGU04LoI+5s=' 'sha256-ipy9P/3rZZW06mTLAR0EnXvxSNcnfSDPLDuh3kzbB1w=' js.bizographics.com https://www.redhat.com assets.adobedtm.com jsonip.com https://ajax.googleapis.com https://www.googletagmanager.com https://www.google-analytics.com https://use.fontawesome.com; style-src 'self' https://fonts.googleapis.com https://use.fontawesome.com; img-src 'self' *; media-src 'self' ; frame-src https://www.googletagmanager.com https://www.youtube.com; frame-ancestors 'none'; base-uri 'none'; object-src 'none'; form-action 'none'; font-src 'self' https://use.fontawesome.com https://fonts.gstatic.com;">
  <META HTTP-EQUIV='X-Frame-Options' CONTENT="DENY">
  <META HTTP-EQUIV='X-XSS-Protection' CONTENT="1; mode=block">
  <META HTTP-EQUIV='X-Content-Type-Options' CONTENT="nosniff">
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="description" content="Quarkus: Supersonic Subatomic Java">
  <meta name="twitter:card" content="summary_large_image">
  <meta name="twitter:site" content="@QuarkusIO"> 
  <meta name="twitter:creator" content="@QuarkusIO">
  <meta property="og:url" content="https://quarkus.io/guides/performance-measure" />
  <meta property="og:title" content="Quarkus - Measuring Performance" />
  <meta property="og:description" content="Quarkus: Supersonic Subatomic Java" />
  <meta property="og:image" content="/assets/images/quarkus_card.png" />
  <link rel="canonical" href="https://quarkus.io/guides/performance-measure">
  <link rel="shortcut icon" type="image/png" href="/favicon.ico" >
  <link rel="stylesheet" href="https://quarkus.io/guides/stylesheet/config.css" />
  <link rel="stylesheet" href="/assets/css/main.css" />
  <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.1.0/css/all.css" integrity="sha384-lKuwvrZot6UHsBSfcMvOkWwlCMgc0TaWr+30HWe3a4ltaBwTZhyTEggF5tJv8tbt" crossorigin="anonymous">
  <link rel="alternate" type="application/rss+xml"  href="https://quarkus.io/feed.xml" title="Quarkus">
  <script src="https://quarkus.io/assets/javascript/goan.js" type="text/javascript"></script>
  <script src="https://quarkus.io/assets/javascript/hl.js" type="text/javascript"></script>
</head>

<body class="guides">
  <!-- Google Tag Manager (noscript) -->
  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-NJWS5L"
  height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
  <!-- End Google Tag Manager (noscript) -->

  <div class="nav-wrapper">
  <div class="grid-wrapper">
    <div class="width-12-12">
      <input type="checkbox" id="checkbox" />
      <nav id="main-nav" class="main-nav">
  <div class="container">
    <div class="logo-wrapper">
      
        <a href="/"><img src="/assets/images/quarkus_logo_horizontal_rgb_600px_reverse.png" class="project-logo" title="Quarkus"></a>
      
    </div>
    <label class="nav-toggle" for="checkbox">
      <i class="fa fa-bars"></i>
    </label>
    <div id="menu" class="menu">
      <span>
        <a href="/get-started/" class="">Get Started</a>
      </span>
      <span>
        <a href="/guides/" class="active">Guides</a>
      </span>
      <span>
        <a href="/community/" class="">Community</a>
      </span>
      <span>
        <a href="/support/" class="">Support</a>
      </span>
      <span>
        <a href="/blog/" class="">Blog</a>
      </span>
      <span>
        <a href="https://code.quarkus.io" class="button-cta secondary white">Start Coding</a>
      </span>
    </div>
  </div>
      </nav>
    </div>
  </div>
</div>

  <div class="content">
    <div class="guide">
  <div class="width-12-12">
    <h1 class="text-caps">Quarkus - Measuring Performance</h1>
    <div class="hide-mobile toc"><ul class="sectlevel1">
<li><a href="#how-do-we-measure-memory-usage">How do we measure memory usage</a>
<ul class="sectlevel2">
<li><a href="#native-memory-tracking">Native Memory Tracking</a></li>
<li><a href="#cloud-native-memory-limits">Cloud Native Memory Limits</a></li>
<li><a href="#platform-specific-memory-reporting">Platform Specific Memory Reporting</a></li>
</ul>
</li>
<li><a href="#how-do-we-measure-startup-time">How do we measure startup time</a></li>
<li><a href="#additional-flags-applied-by-quarkus">Additional flags applied by Quarkus</a>
<ul class="sectlevel2">
<li><a href="#disable-fallback-images">Disable fallback images</a></li>
<li><a href="#disable-isolates">Disable Isolates</a></li>
<li><a href="#disable-auto-registration-of-all-service-loader-implementations">Disable auto-registration of all Service Loader implementations</a></li>
<li><a href="#better-default-for-garbage-collection-implementation">Better default for Garbage Collection implementation</a></li>
<li><a href="#others">Others &#8230;&#8203;</a></li>
</ul>
</li>
</ul></div>
    <div>
      <div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This guide covers:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>how we measure memory usage</p>
</li>
<li>
<p>how we measure startup time</p>
</li>
<li>
<p>which additional flags will Quarkus apply to native-image by default</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>All of our tests are run on the same hardware for a given batch.
It goes without saying but it&#8217;s better when you say it.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="how-do-we-measure-memory-usage"><a class="anchor" href="#how-do-we-measure-memory-usage"></a>How do we measure memory usage</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When measuring the footprint of a Quarkus application, we measure <a href="https://en.wikipedia.org/wiki/Resident_set_size">Resident Set Size (RSS)</a>
and not the JVM heap size which is only a small part of the overall problem.
The JVM not only allocates native memory for heap (<code>-Xms</code>, <code>-Xmx</code>) but also structures required by the jvm to run your application. Depending on the JVM implementation, the total memory allocated for an application will include, but not limited to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Heap space</p>
</li>
<li>
<p>Class metadata</p>
</li>
<li>
<p>Thread stacks</p>
</li>
<li>
<p>Compiled code</p>
</li>
<li>
<p>Garbage collection</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="native-memory-tracking"><a class="anchor" href="#native-memory-tracking"></a>Native Memory Tracking</h3>
<div class="paragraph">
<p>In order to view the native memory used by the JVM, you can enable the <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/vm/nmt-8.html">Native Memory Tracking</a> (NMT) feature in hotspot;</p>
</div>
<div class="paragraph">
<p>Enable NMT on the command line;</p>
</div>
<div class="literalblock">
<div class="content">
<pre>-XX:NativeMemoryTracking=[off | summary | detail] <i class="conum" data-value="1"></i><b>(1)</b></pre>
</div>
</div>
<div class="colist arabic">
<table>
<tr>
<td><i class="conum" data-value="1"></i><b>1</b></td>
<td>NOTE: this feature will add cause an approximately 5-10% performance overhead</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>It is then possible to use jcmd to dump a report of the native memory usage of the Hotspot JVM running your application;</p>
</div>
<div class="literalblock">
<div class="content">
<pre>jcmd &lt;pid&gt; VM.native_memory [summary | detail | baseline | summary.diff | detail.diff | shutdown] [scale= KB | MB | GB]</pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="cloud-native-memory-limits"><a class="anchor" href="#cloud-native-memory-limits"></a>Cloud Native Memory Limits</h3>
<div class="paragraph">
<p>It is important to measure the whole memory to see the impact of a Cloud Native application.
It is particularly true of container environments which will kill a process based on its full RSS memory usage.</p>
</div>
<div class="paragraph">
<p>Likewise, don&#8217;t fall into the trap of only measuring private memory which is what the process uses that is not shareable with other processes.
While private memory might be useful in a environment deploying many different applications (and thus sharing memory a lot),
it is very misleading in environments like Kubernetes/OpenShift.</p>
</div>
</div>
<div class="sect2">
<h3 id="platform-specific-memory-reporting"><a class="anchor" href="#platform-specific-memory-reporting"></a>Platform Specific Memory Reporting</h3>
<div class="paragraph">
<p>In order to not incur the performance overhead of running with NVM enabled, we measure the total RSS of an JVM application using tools specific to each platform.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Linux</dt>
<dd>
<p>The linux <a href="https://linux.die.net/man/1/pmap">pmap</a> and <a href="https://linux.die.net/man/1/ps">ps</a> tools provide a report on the native memory map for a process</p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="shell" class="language-shell hljs"> $ ps -o pid,rss,command -p &lt;pid&gt;

   PID   RSS COMMAND
 11229 12628 ./target/getting-started-1.0-SNAPSHOT-runner</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="shell" class="language-shell hljs"> $ pmap -x &lt;pid&gt;

 13150:   /data/quarkus-application -Xmx100m -Xmn70m
 Address           Kbytes     RSS   Dirty Mode  Mapping
 0000000000400000   55652   30592       0 r-x-- quarkus-application
 0000000003c58000       4       4       4 r-x-- quarkus-application
 0000000003c59000    5192    4628     748 rwx-- quarkus-application
 00000000054c0000     912     156     156 rwx--   [ anon ]
 ...
 00007fcd13400000    1024    1024    1024 rwx--   [ anon ]
 ...
 00007fcd13952000       8       4       0 r-x-- libfreebl3.so
 ...
 ---------------- ------- ------- -------
 total kB         9726508  256092  220900</code></pre>
</div>
</div>
<div class="paragraph">
<p>Each Memory region that has been allocated for the process is listed;</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Address: Start address of virtual address space</p>
</li>
<li>
<p>Kbytes:  Size (kilobytes) of virtual address space reserved for region</p>
</li>
<li>
<p>RSS:     Resident set size (kilobytes).  This is the measure of how much memory space is actually being used</p>
</li>
<li>
<p>Dirty:   dirty pages (both shared and private) in kilobytes</p>
</li>
<li>
<p>Mode:    Access mode for memory region</p>
</li>
<li>
<p>Mapping: Includes application regions and Shared Object (.so) mappings for process</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The Total RSS (kB) line reports the total native memory the process is using.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">macOS</dt>
<dd>
<p>On macOS, you can use <code>ps x -o pid,rss,command -p &lt;PID&gt;</code> which list the RSS for a given process in KB (1024 bytes).</p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="shell" class="language-shell hljs">$ ps x -o pid,rss,command -p 57160

  PID    RSS COMMAND
57160 288548 /Applications/IntelliJ IDEA CE.app/Contents/jdk/Contents/Home/jre/bin/java</code></pre>
</div>
</div>
<div class="paragraph">
<p>Which means IntelliJ IDEA consumes 281,8 MB of resident memory.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="how-do-we-measure-startup-time"><a class="anchor" href="#how-do-we-measure-startup-time"></a>How do we measure startup time</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Some frameworks use aggressive lazy initialization techniques.
It is important to measure the startup time to first request to most accurately reflect how long a framework needs to start.
Otherwise, you will miss the time the framework <em>actually</em> takes to initialize.</p>
</div>
<div class="paragraph">
<p>Here is how we measure startup time in our tests.</p>
</div>
<div class="paragraph">
<p>We create a sample application that logs timestamps for certain points in the application lifecycle.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="java" class="language-java hljs">@Path("/")
public class GreetingEndpoint {

    private static final String template = "Hello, %s!";

    @GET
    @Path("/greeting")
    @Produces("application/json")
    public Greeting greeting(@QueryParam("name") String name) {
        System.out.println(new SimpleDateFormat("HH:mm:ss.SSS").format(new java.util.Date(System.currentTimeMillis())));
        String suffix = name != null ? name : "World";
        return new Greeting(String.format(template, suffix));
    }

    void onStart(@Observes StartupEvent startup) {
        System.out.println(new SimpleDateFormat("HH:mm:ss.SSS").format(new Date()));
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We start looping in a shell, sending requests to the rest endpoint of the sample application we are testing.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="shell" class="language-shell hljs">$ while [[ "$(curl -s -o /dev/null -w ''%{http_code}'' localhost:8080/api/greeting)" != "200" ]]; do sleep .00001; done</code></pre>
</div>
</div>
<div class="paragraph">
<p>In a separate terminal, we start the timing application that we are testing, printing the time the application starts</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code data-lang="shell" class="language-shell hljs">$ date +"%T.%3N" &amp;&amp;  ./target/quarkus-timing-runner

10:57:32.508
10:57:32.512
2019-04-05 10:57:32,512 INFO  [io.quarkus] (main) Quarkus 0.11.0 started in 0.002s. Listening on: http://127.0.0.1:8080
2019-04-05 10:57:32,512 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, resteasy-jsonb]
10:57:32.537</code></pre>
</div>
</div>
<div class="paragraph">
<p>The difference between the final timestamp and the first timestamp is the total startup time for the application to serve the first request.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="additional-flags-applied-by-quarkus"><a class="anchor" href="#additional-flags-applied-by-quarkus"></a>Additional flags applied by Quarkus</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When Quarkus invokes GraalVM <code>native-image</code> it will apply some additional flags by default.</p>
</div>
<div class="paragraph">
<p>You might want to know about the following ones in case you&#8217;re comparing performance properties with other builds.</p>
</div>
<div class="sect2">
<h3 id="disable-fallback-images"><a class="anchor" href="#disable-fallback-images"></a>Disable fallback images</h3>
<div class="paragraph">
<p>Fallback native images are a feature of GraalVM to "fall back" to run your application in the normal JVM, should the compilation
to native code fail for some reason.</p>
</div>
<div class="paragraph">
<p>Quarkus disables this feature by setting <code>-H:FallbackThreshold=0</code>: this will ensure you get a compilation failure rather
risking to not notice that the application is unable to really run in native mode.</p>
</div>
<div class="paragraph">
<p>If you instead want to just run in Java mode, that&#8217;s totally possible: just skip the native-image build and run it as a jar.</p>
</div>
</div>
<div class="sect2">
<h3 id="disable-isolates"><a class="anchor" href="#disable-isolates"></a>Disable Isolates</h3>
<div class="paragraph">
<p>Isolates are a neat feature of GraalVM, but Quarkus isn&#8217;t using them at this stage.</p>
</div>
<div class="paragraph">
<p>Disable via <code>-H:-SpawnIsolates</code>.</p>
</div>
</div>
<div class="sect2">
<h3 id="disable-auto-registration-of-all-service-loader-implementations"><a class="anchor" href="#disable-auto-registration-of-all-service-loader-implementations"></a>Disable auto-registration of all Service Loader implementations</h3>
<div class="paragraph">
<p>Quarkus extensions can automatically pick the right services they need, while GraalVM&#8217;s native-image defaults to include
all services it&#8217;s able to find on the classpath.</p>
</div>
<div class="paragraph">
<p>We prefer listing services explicitly as it produces better optimised binaries. Disable it as well by setting <code>-H:-UseServiceLoaderFeature</code>.</p>
</div>
</div>
<div class="sect2">
<h3 id="better-default-for-garbage-collection-implementation"><a class="anchor" href="#better-default-for-garbage-collection-implementation"></a>Better default for Garbage Collection implementation</h3>
<div class="paragraph">
<p>The default in GraalVM seems meant to optimise for short lived processes.</p>
</div>
<div class="paragraph">
<p>Quarkus defaults to server applications, so we switch to a better default by setting
 <code>-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime</code>.</p>
</div>
</div>
<div class="sect2">
<h3 id="others"><a class="anchor" href="#others"></a>Others &#8230;&#8203;</h3>
<div class="paragraph">
<p>This section is provided as high level guidance, but can&#8217;t presume to be comprehensive as some flags are controlled
 dynamically by the extensions, the platform you&#8217;re building on, configuration details, your code and possibly
 a combination of any of these.</p>
</div>
<div class="paragraph">
<p>Generally speaking the ones listed here are those most likely to affect performance metrics, but in the right
circumstances one could observe non negligible impact from the other flags too.</p>
</div>
<div class="paragraph">
<p>If you&#8217;re to investigate some differences in detail make sure to check what Quarkus is invoking exactly: when the build
plugin is producing a native image, the full command lines are logged.</p>
</div>
</div>
</div>
</div>
    </div>
  </div>
</div>

  </div>

  <div class="content project-footer">
  <div class="footer-section">
    <div class="logo-wrapper">
      <a href="/"><img src="/assets/images/quarkus_logo_horizontal_rgb_reverse.svg" class="project-logo" title="Quarkus"></a>
    </div>
  </div>
  <div class="grid-wrapper">
    <p class="grid__item width-3-12">Quarkus is open. All dependencies of this project are available under the <a href='https://www.apache.org/licenses/LICENSE-2.0' target='_blank'>Apache Software License 2.0</a> or compatible license.<br /><br />This website was built with <a href='https://jekyllrb.com/' target='_blank'>Jekyll</a>, is hosted on <a href='https://pages.github.com/' target='_blank'>Github Pages</a> and is completely open source. If you want to make it better, <a href='https://github.com/quarkusio/quarkusio.github.io' target='_blank'>fork the website</a> and show us what you’ve got.</p>

    
      <div class="width-1-12 project-links">
        <span>Navigation</span>
        <ul class="footer-links width-1-12">
          
            <li><a href="/">Home</a></li>
          
            <li><a href="/guides">Guides</a></li>
          
            <li><a href="/community/#contributing">Contribute</a></li>
          
            <li><a href="/faq">FAQ</a></li>
          
            <li><a href="/get-started">Get Started</a></li>
          
        </ul>
      </div>
    
      <div class="width-1-12 project-links">
        <span>Contribute</span>
        <ul class="footer-links width-1-12">
          
            <li><a href="https://twitter.com/quarkusio">Follow us</a></li>
          
            <li><a href="https://github.com/quarkusio">GitHub</a></li>
          
            <li><a href="/security">Security&nbsp;policy</a></li>
          
        </ul>
      </div>
    
      <div class="width-1-12 project-links">
        <span>Get Help</span>
        <ul class="footer-links width-1-12">
          
            <li><a href="https://groups.google.com/forum/#!forum/quarkus-dev">Forums</a></li>
          
            <li><a href="https://quarkusio.zulipchat.com">Chatroom</a></li>
          
        </ul>
      </div>
    

    
      <div class="width-3-12 more-links">
        <span>Quarkus is made of community projects</span>
        <ul class="footer-links">
          
            <li><a href="https://vertx.io/" target="_blank">Eclipse Vert.x</a></li>
          
            <li><a href="https://microprofile.io" target="_blank">Eclipse MicroProfile</a></li>
          
            <li><a href="https://hibernate.org" target="_blank">Hibernate</a></li>
          
            <li><a href="https://netty.io" target="_blank">Netty</a></li>
          
            <li><a href="https://resteasy.github.io" target="_blank">RESTEasy</a></li>
          
            <li><a href="https://camel.apache.org" target="_blank">Apache Camel</a></li>
          
            <li><a href="https://code.quarkus.io/" target="_blank">And many more...</a></li>
          
        </ul>
      </div>
    
  </div>
</div>
  <div class="content redhat-footer">
  <div class="grid-wrapper">
    <span class="licence">
      <i class="fab fa-creative-commons"></i><i class="fab fa-creative-commons-by"></i> <a href="https://creativecommons.org/licenses/by/3.0/" target="_blank">CC by 3.0</a> | <a href="https://www.redhat.com/en/about/privacy-policy">Privacy Policy</a>
    </span>
    <span class="redhat">
      Sponsored by
    </span>
    <span class="redhat-logo">
      <a href="https://www.redhat.com/" target="_blank"><img src="/assets/images/redhat_reversed.svg"></a>
    </span>
  </div>
</div>


  <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.4/jquery.min.js" integrity="sha384-8gBf6Y4YYq7Jx97PIqmTwLPin4hxIzQw5aDmUg/DDhul9fFpbbLcLh3nTIIDJKhx" crossorigin="anonymous"></script>
  <script type="text/javascript" src="/assets/javascript/mobile-nav.js"></script>
  <script type="text/javascript" src="/assets/javascript/scroll-down.js"></script>
  <script src="/assets/javascript/satellite.js" type="text/javascript"></script>
  <script src="https://quarkus.io/guides/javascript/config.js" type="text/javascript"></script>
  <script src="/assets/javascript/search-filter.js" type="text/javascript"></script>
  <script src="/assets/javascript/back-to-top.js" type="text/javascript"></script>
</body>

</html>
